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Abstract. We construct generating trees with one, two, and three labels for some classes of permuta- 
tions avoiding generalized patterns of length 3 and 4. These trees are built by adding at each level an 
entry to the right end of the permutation, which allows us to incorporate the adjacency condition about 
some entries in an occurrence of a generalized pattern. We use these trees to find functional equations 
for the generating functions enumerating these classes of permutations with respect to different param- 
eters. In several cases we solve them using the kernel method and some ideas of Bousquet-Melou [4]. 
We obtain refinements of known enumerative results and find new ones. 

1. Introduction 

1.1. Generalized pattern avoidance. We denote by S n the symmetric group on {1,2,..., n}. Let n 
and k be two positive integers with k < n, and let tt = 7Ti7T2 • ■ ■ n n £ S n be a permutation. A generalized 
pattern a is obtained from a permutation o\02 ■ ■ • &k £ Sk by choosing, for each j = I, . . . ,k — 1, either 
to insert a dash - between <jj and (7j+i or not. More formally, a — o~\E\o-iEi • ■ 'Sk-iCfc, where each Ej 
is either the symbol - or the empty string. With this notation, we say that tt contains (the generalized 
pattern) a if there exist indices i\ < i-i < ■ ■ ■ < ik such that (i) for each j — 1, . . . , k — 1, if Sj is empty 
then ij+i = in + 1, and (ii) for every a, 6 £ {1, 2, . . . , fc}, n ia < ir ih if and only if a a < a^. In this case, 
^i 1 ^i 2 ' ' ' ^ik i s called an occurrence of a in tt. 

If tt does not contain a, we say that tt avoids a, or that it is a-avoiding. For example, the permutation 
tt = 3542716 contains the pattern 12-4-3 because it has the subsequence 3576. On the other hand, tt 
avoids the pattern 12-43. We denote by S n (a) the set of permutations in S n that avoid a. More generally, 
if £ = {ci, (72, . . .} is a collection of generalized patterns, we say that a permutation tt is E-avoiding if tt 
is er-avoiding for all a G X. We denote by <S n (E) the set of E-avoiding permutations in S n - 

We use the word length to refer to the number of letters in a permutation, so that S n is the set of 
permutations of length n. A class will consist of a set (e.g., all permutations avoiding a given pattern) 
together with a function (e.g., the length). Given a permutation tt £ S n , we will write r(7r) = 7r„ to 
denote the rightmost entry of tt. In all our generating functions, the variable t will mark the length of 
the permutation. 

1.2. Generating trees. Generating trees are a useful tool for enumerating classes of pattern-avoiding 
permutations (see, for example, jTTl [18] ) . The nodes at each level of the generating tree are indexed by 
permutations of a given length. It is common in the literature to define the children of a permutation n of 
length n to be those permutations that are obtained by inserting the entry n+1 in tt = ttykz • • • 7T n in such 
a way that the new permutation is still in the class. In this paper we consider a variation of this definition. 
Here, the children of a permutation tt of length n are obtained by appending an entry to the right of tt, 
and adding one to all the entries in tt that were greater than or equal to the new entry. For example, if 
the entry 3 is appended to the right of tt — 24135, the child that we obtain is 251463. Adding the new 
entry to the right of the permutation makes these trees well-suited to enumerate permutations avoiding 
generalized patterns, as we will see throughout the paper. We will refer to these trees as rightward 
generating trees. This kind of generating trees has been used in [3] to enumerate permutations avoiding 
sets of three generalized patterns of length three with one dash, such as {1-23, 2-13, 1-32}. 

For some classes of permutations, a label (£) can be associated to each node of the tree in such a 
way that the number of children of a permutation and their labels depend only on the label of the 
parent. For example, in the tree for 1-2-3-avoiding permutations, we can label each node tt with m = 

l 
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min{7Ti : 3j < i with wj < iti} (or m = n + 1 if tt = n ■ ■ ■ 21). Then, the children of a permutation with 
label (to) have labels (to + 1), (2), (3), . . . , (m), corresponding to the appended entry being 1, 2, 3, . . . , to, 
respectively. This succession rule, together with the fact that the root (tt = 1 € Si) has label (2), 
completely determines the tree. From this rule one can derive a functional equation for the generating 
function that enumerates the permutations by their length and the label of the corresponding node in the 
tree for this class of 1-2-3-avoiding permutations. For generating trees with one label, these equations are 
well understood and their solutions are algebraic series. This is the case of the generating trees obtained 
in [3J, for example. 

In other cases, however, one label is not enough to describe the generating tree in terms of a succession 
rule. Generating trees with two labels were used in [3] to enumerate restricted permutations. In fact, 
the inspiration for the present paper and many of the ideas used come from Bousquet-Melou's work. 
One difference is that here trees are constructed by adding at each level an entry to the right end of the 
permutation, which allows us to keep track of elements occurring in adjacent positions. In Section [4] we 
consider some classes of permutations whose rightward generating tree has three labels for each node. 



1.3. Organization of the paper. In this paper we enumerate several families of permutations that 
avoid generalized patterns. What ties together the results in the different sections is the technique that 
we use to obtain them. The strategy consists of building a rightward generating tree for the family of 
permutations, translating the succession rule into a set of functional equations, and applying the kernel 
method to them. We have tried this strategy for a number of classes of permutations, and we have found 
it to work in several cases, which we include here. This is why the sets of generalized patterns that we 
discuss may seem somewhat arbitrary. For other patterns one can construct similar generating trees with 
two or three labels, but we have not been able to solve the corresponding functional equations for the 
generating function, so we have not included these examples here. In any event, this paper is not meant 
to be an exhaustive study of the sets of patterns for which this technique would work. 

In general, we have looked for sets of patterns for which the rightward generating tree of the class of 
permutations avoiding them has a simple succession rule, once appropriate labels are chosen. In some 
cases, we have chosen patterns based on the elegance of their enumerating sequence, like in Section ^. 21 In 
others, we have chosen patterns whose corresponding generating function has zero radius of convergence, 
as is the case in Sections 13.61 14.11 and 14.21 These seem to be the first instances of generating functions 
with zero radius of convergence that arise from generating trees and the kernel method. 

We have classified the sets of studied patterns depending on how many labels are needed to describe the 
generating tree. In Section [2] we consider some families of permutations where the tree can be described 
with one label, which is the value of the rightmost entry in the permutation. The results in this section 
are new, and all involve permutations that avoid the pattern 2-1-3. This makes the succession rules easier 
because this restriction prevents a permutation with rightmost value r to have a child with rightmost 
value greater than r + 1 . 

In Section [3] we study classes of permutations where each node of the generating tree bears a pair of 
labels. For most of them we get rational or algebraic generating functions, and their enumeration has 
been done in the literature using different techniques. Section [4] contains some of the main results of the 
paper. We find ordinary generating functions for {1-23, 3-12, 34-21}-avoiding and {1-23, 34-21 }-avoiding 
permutations. Both families are described by generating trees with three labels. 

Additional motivation for the study of these families of permutations comes from trying to under- 
stand the possible asymptotic behaviors of the number of permutations avoiding generalized patterns 
(see [10]). An asymptotic analysis of the coefficients of the generating functions for {1-23, 3-12}-avoiding, 
{1-23, 34-21 {-avoiding, and {1-23, 3-12, 34-21}-avoiding permutations that we have found may reveal that 
their asymptotic growth is strictly smaller than that of Bell numbers but strictly greater than exponen- 
tial. This would be the first known instance of a family of pattern-avoiding permutations that exhibits 
such a behavior. 
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2. Generating trees with one label 

In this section we enumerate classes of pattern-avoiding permutations whose rightward generating trees 
can be described by a succession rule involving only one label for each node. For some of these classes, 
rightward generating trees are not the only way to obtain the results, but they are a tool that works in 
all these cases. 

The classes in this section avoid the pattern 2-1-3. Note that avoiding this pattern is equivalent to 
avoiding the generalized pattern 2-13. Indeed, if it contains an occurrence of 2-1-3, say iTiirjiik with 
nj < ir-i < 7Tfc, then there must be some index £ with j < i < k such that tt£ < 7Tj and 7r^ +1 > 7r.j, so 
TTiireTte+i is an occurrence of 2-13. For any class of permutations that avoid this pattern, the corresponding 
rightward generating tree has the property that the appended entry at each level can never be more than 
one unit larger than the entry appended at the previous level. 

2.1. {2-1-3, 2-31}-avoiding permutations. A permutation it is said to avoid the barred pattern 2-31 
if every descent in tt (an occurrence of the generalized pattern 21) is part of an occurrence of 2-31; 
equivalently, for any index i such that iti > 7Ti + i there is an index j < i such that Hi > Ttj > i^i+i- The 
bar indicates that the 2 is forced whenever a 31 occurs. For example, the permutation 4627513 avoids 
2-31, but 2475613 does not. 

We use M n to denote the n-th Motzkin number. Recall that Y, n >o M nt n = 1 ^ t ^ v ^ 2t ^ —. The next 
result seems to be a new interpretation of the Motzkin numbers. 

Proposition 2.1. The number of {2-1-3, 2-31} -avoiding permutations of size n is M„_i. 

Proof. Consider the rightward generating tree for {2-1-3, 2-31}-avoiding permutations. Labeling each 
permutation with its rightmost entry r = r(7r), this tree is described by the succession rule 

(1) 

(r)— >(1) (2) ••• (r-1) (r + 1). 

Indeed, the new entry appended to the right of n cannot be greater than n n + 1 in order for the new 
permutation to be 2-1-3-avoiding, and it cannot be 7r n because then it would create an occurrence of 21 
that is not part of an occurrence of 2-31. 

Defining D(t, u) = ^ n>1 J2nes n (2-i-3 2-31) uT^H™, the succession rule above gives the following equa- 
tion for the generating function: 

(1) (l tu] D(t,u) =tu —D(t,l). 

\ u-1 J u — 1 

The next step is to apply the kernel method. This technique, which has been part of mathematical 

folklore for decades, has recently been systematized in [TJ ® [5] . Of the two values of u as a function 

of t that cancel the term multiplying D(t,u) on the left hand side, uo = uo(t) = 1 + t -^ 1 ^~ 2t - 3t ' 2 [ s a 

well-defined formal power series in t. Substituting u — uq in (fT]) gives 



, . 1 - t - Vl - 2* - 3i 2 
D(t, 1) = uo - 1 = — f , 

which is the generating function for the Motzkin numbers with the indices shifted by one. □ 

There is also a bijective proof of Proposition ! 2.11 Given apermutation 7r = 7Ti7T2 ■ ■ • 7r„ S <S n (2-l-3, 2-31), 
we can construct a Dyck path of size n (i.e., a sequence of n Us and n Ds so that no prefix contains more 
Ds than Us) as follows. A right-to-left maximum of it is an entry 7Tj such that 7Tj > 7Tj for all j > i. Let 
7!"^, 7Tj 2 , . . . , TTi m be the right-to-left maxima of n, with i\ < ii < • • • < i m = n. Consider the Dyck path 

tp(ir) = U ix D' Kil ~' Ki " 2 U i ' 1 '~ ix D' Ki i~' Ki zU i *~ i ' i ■ ■ ■ D Wi ">-i-~ Wim U im ~ im - 1 D J * i ™ , 

where exponentiation indicates repetition of a step. This map is a bijection between 2-1-3-avoiding 
permutations and Dyck paths (see [33]), and it is not hard to see that the condition if it being 2-31- 
avoiding is equivalent to the requirement that the path contains no three consecutive steps UDU. So, 
we have a bijection between S n (2-1-3, 2-31) and UDU-hee Dyck paths of size n. 
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To finish the proof, we next describe a bijection due to Callan [7] between UDU-hee Dyck paths of 
size n and Motzkin paths of length n — 1 (i.e., sequences of n — 1 steps U, D, and H with the same 
number of Us and Ds and so that no prefix contains more Ds than ?7s) . We say that a U and aflina 
Dyck path are matched if the D is to the right of the U and the letters between them form a Dyck path. 
Note that each step is matched with exactly another one. Given a U DU-iiee Dyck path, first append a 
D to it. Now, for each D that is immediately preceded and followed by D steps, delete it and replace 
its matching U with an H. Next, replace each occurrence of UDD with a D. Finally, delete the D that 
was appended to the path. This produces a Motzkin path of length n, — 1. The composition of these two 
bijections completes the bijective proof of Proposition l2.il 

2.2. {2-1-3, 2°-31}-avoiding permutations. Extending the notion of barred patterns, we say that a 
permutation tt avoids the pattern 2 -31 if every descent in 7r is the '31' part of an odd number of 
occurrences of 2-31; equivalently, for any index i such that 7rj > iti+i, the number of indices j < i such 
that TTi > TTj > iti+i is odd. 



Proposition 2.2. The number of {2-1-3, 2 -31} -avoiding permutations of size n is 

( 1 (3k\ 

ifn = 2k + l. 



(2) |5 n (2-l-3,2°-31)| = ^Yi) 3 ^ lfn 2fc ' 



I, 2fe+l V fc+1 , 

Proof. The rightward generating tree for {2-1-3, 2°-31}-avoiding permutations is given by the succession 
rule 

(1) 

(r) — ■■•(»•- 3) (r-1) (r + 1), 

that is, the labels of the children of a node labeled r are the numbers 1 < j < r + 1 such that r — j is 
odd. Let J(t,u) = £„>! E* e 5 f ,(2-i-3,r-3i) urMfn = Sr>i "M*K, and let J%t,u) = J2 r cvcn J r (t)u r . 
The succession rule translates into the following functional equation: 

/ s / tu 3 \ , , tu 2 , . tu(u— 11 „. 

(3) 1 - J(t, u)=tU- ~ 1 —rJ(t, 1) + l 2 > J e (t, 1). 

\ vr — 1 / u — 1 u — 1 

The kernel 1 — -tzt as a function in the variable u has three zeroes, two of which are complex conjugates. 
Denote them by u% = a(t) + b(t)i and U2 = u\ = a(t) — b(t)i. Adding the equations = uf — l — UiJ(t, 1) + 
(ui - l)J e {t, 1) for i = 1,2, we get 

a(t)J(t, 1) = a(t) 2 - b(t) 2 - 1 + (a(t) - l)J e (t, 1), 

and subtracting them gives 

J(t,l) = 2a(t) + J e (t,l). 

Solving this system of equations for J, we get that J(t, 1) = 2a(t) — a(t) 2 — b(t) 2 — 1. Plugging in the 
values of a(t) and b(t) yields the expression 

(2 - 3t)f(t) 2 + (9t~2- g{t))f{t) + (2 - 6t)g(t) + 54t 2 - lgt-4 
1 '~ 3tf(t) 2 

where gr(i) = ^3(27i 2 - 4) and /(<) = [12tg(t) - 108i 2 + 8] 1/3 . It is easy to check that J = J(t, 1) is a 
root of the polynomial t J 3 + (3t — 2) J 2 + (3i — 1) J + t = 0. Using the Lagrange inversion formula, one 
sees that its coefficients are given by ([2]), which is sequence A047749 from the On-Line Encyclopedia of 
Integer Sequences [16]. Observe that we can also obtain an expression for J{t,u) using ((3]) and the fact 
that J e (i, 1) = -a{t) 2 - b(t) 2 - 1. □ 



It is also possible to give a direct bijective proof of Proposition 12.21 that does not use rightward 
generating trees. A well-known combinatorial interpretation of the numbers (|2|) is that they enumerate 
lattice paths from (0,0) to (n, [n/2\) with steps E = (1,0) and N — (0,1) that never go above the 
line y = x/2. We next describe a bijection from <S„(2-l-3, 2 -31) to these paths. Let 7r = 7ri7r 2 • • • n n £ 
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S n (2-1-3, 2°-31). Let n^, 7Tj 2 , . . . , TTi m be the right-to-left maxima of tt, with i\ < i% < ■ ■ ■ < i m = n. We 
claim that the condition that tt is {2-1-3, 2 -31 {-avoiding guarantees that all the differences tt^ — 7Tj +1 
are even. To see this, fix j and let O be the set of entries a such that cnr^ 71-^+1 is an occurrence of 2-31. 
Since tt avoids 2°-31, the cardinality of O is odd. Now, every a € O must satisfy a > 7Ti- +1 . This is 
obvious if ij + 1 = and otherwise it follows from the fact that if a < 7r.; j+1 , then a r Ki j+ iiTi i+1 would 
be an occurrence of 2-1-3. On the other hand, any entry a with TTi j+1 < a < 71",^. must appear to the left of 
TTi j , since tt^ and TTi j+1 are consecutive right-to-left maxima, and so a € O. Thus, O is precisely the set of 
integers strictly between and 7r.j j+1 , which implies that 71^ — 7Tj j+1 is even. Now, for j = 1, . . . , m — 1, 
let a,j — (n ij — n ij+1 )/2. Let a m — [n im /2\. We map tt to the following path from (0,0) to (n, [n/2j): 

It can be checked that this map is a bijection. For example, if tt — 4675123, we have T^Tr^Tr^ = 773^4^7 = 
753, so the corresponding path from (0, 0) to (7, 3) is EEENENEEEN. 

Aside from lattice paths, the sequence d n :— |6>„(2-l-3, 2°-31)| from ([2]) is also known to enumerate 
symmetric ternary trees with 3n edges and symmetric diagonally convex directed polyominoes of area n. 
These numbers have also appeared before in connection to pattern- avoiding permutations. It is shown 
in [6] that the number of 2143-avoiding Dumont permutations of the second kind of length 2n is d n d n +i 
(see [B] for definitions). The sequence d n enumerates what the authors call lower boards, which are 
2-1-3-avoiding permutations of length n whose diagram fits in a certain shape. A bijection between such 
permutations and S n (2-1-3, 2 -31) can be established by composing our bijection into lattice paths with 
the one from [6]. 

Analogously to the definition for the pattern 2°-31, we say that a permutation n avoids the pattern 
2 -31 if every occurrence of 21 in tt is part of an even number of occurrences of 2-31. We can also 
enumerate {2-1-3, 2 -31{-avoiding permutations. 

Proposition 2.3. The number of {2-1-3, 2 e -31{ -avoiding permutations of size n is 



, L«/2J 

- E 

k=Q 



n \ n 



— k\ n ( n \ I n — k 



2k \k,-l n- k\2k + lj \ k 



Proof. Let Q(t) = J2 n >i (2-1-3, 2 e -31)| t n . An argument similar to the proof of Proposition 12 . 21 shows 
that 

_ (2 - At)f{tf + (-2 + \2t - 7t 2 - g(t))f(t) + (2 - 8t)g(t) + 8t 3 + 46t 2 - 8t - 4 
~ 3t/(<) 2 



where g(t) = ^3(-5t 4 + 24t 3 - At 2 + 12t - 4) and f(t) = [A(3tg(t) - lit 3 - 12t 2 - 6t + 2)] 1 / 3 . It follows 
that Q — Q(t) is a root of the polynomial tQ 3 + (At — 2)Q 2 + (At — 1)Q + 1 = 0. Applying Lagrange 
inversion we get the stated formula. □ 

2.3. {2-1-3, 2-3-41, 3-2-41{-avoiding permutations. The rightward generating tree for this class of 
permutations has a simple succession rule. This allows us to enumerate them easily. Let K(t,u) — 

En>l S7re5„(2-l-3, 2-3-41,3-2-41) U r n t n — Ylr>l K r (t)u r . 

Proposition 2.4. The generating function for {2-1-3, 2-3-41, 3-2 -Al}-avoiding permutations where u 
marks the value of the rightmost entry is 



„, . 1 - t - 2tu - Vl - 2< - 'it 2 

Kit.u) = — . 

1 ^ 2t(i + l + u)-2 

Proof. The succession rule for this class of permutations is 



(r) 



(1)(2) ifr=l, 
(r - 1) (r) (r + 1) if r > 1, 
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with the root labeled (1). This translates into the functional equation 



(4) 



I - / | - + 1 + u 

u 



K(t,u) =tu-tK x {t). 



Applying the kernel method we find that Ki(t) — — 1 ^ 2t 1 , and substituting back into (j4|) we get 
the expression for K(t, u). □ 

The generating function K(t, 1) also enumerates {1-3-2, 123-4}-avoiding permutations, as shown in p~5j 
Example 2.6]. However, no direct bijection between S„(2-l-3, 2-3-41, 3-2-41) and S n (1-3-2, 123-4) seems 
to be known. 

3. Generating trees with two labels 

The generating trees in all the examples in the previous section were described using one label for 
each node. This will not be the case in the families of permutations in this section. However, we will use 
the same technique of translating the succession rule into a set of functional equations and applying the 
kernel method to them. This method is what unifies the different classes of permutations studied in this 
paper. 

Here we enumerate some classes of permutations whose rightward generating tree has a succession rule 
that can be described using a pair of labels for each node. These trees give rise to functional equations 
with three variables. Even though no method is known to solve them in general, in this section we present 
special cases where we have been able to solve the corresponding equations. 

In a few cases, one of the two labels is the length of the permutation. When that happens, the 
functional equations have only two variables, but the variable t appears multiplied by another variable, 
which makes these equations more difficult than the ones in Section [2l 

Note that for the classes that we consider in this section, the enumeration of the permutations by their 
length has already been done by different authors [HI M IHJ HH HS1 HE] • Our contribution is a refined 
enumeration of these permutations by several parameters, and also the fact that our results are obtained 
using the unifying framework of rightward generating trees. 

3.1. {2-1-3, 12-3}-avoiding permutations. It was shown in [8] that |<S„(l-3-2, 1-23) | = M n . A bijection 
between S n (1-3-2, 1-23) and the set of Motzkin paths of length n was given in [11]. Clearly the sets 
6>„(l-3-2, 1-23) and S n (2-l-3, 12-3) are equinumerous, since a permutation 7ri7T2---7r„ is {1-3-2,1-23}- 
avoiding exactly when (n + 1 — 7r„) ■ • • (n + 1 — 7T2)(n + 1 — tti) is {2-1-3, 12-3}-avoiding. In this section we 
recover the formula for |6>„(2-l-3, 12-3) | using a generating tree with two labels. This method provides a 
refined enumeration of {2-1-3, 12-3}-avoiding permutations by two new parameters: the value of the last 
entry and the smallest value of the top of an ascent. 

Let Ti be the rightward generating tree for the set of {2-1-3, 12-3}-avoiding permutations. Given any 
7r e S n , define the parameter 



(5) £(tt) = 



n+1 if Tr = n(n- 1)---21, 

min{7Ti : i > 1, tt^i < iti] otherwise. 



Let each permutation tt be labeled by the pair (£, r) — (((tt), r(jr)). Note that since 7r avoids 12-3, 
then necessarily £ > r. 

Lemma 3.1. The rightward generating tree T\ for {2-1-3, 12 -3} -avoiding permutations is specified by the 
following succession rule on the labels: 

(2,1) 

f(£ + l,l) (£ + 1,2) ••• (i + l,£) if£ = r, 

' } (£ + 1,1) (£+1,2) ••• (£+l,r) (r + l,r + l) if I > r. 



(£,r) 
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Proof. The permutation obtained by appending an entry to the right of ir 6 S n (2-1-3, 12-3) is 2-1-3- 
avoiding if and only if the appended entry is at most r(jr) + 1, and it is 12-3-avoiding if and only if the 
appended entry is at most £(ir). The labels of the children are obtained by looking at how the values of 
(£, r) change when the new entry is added. □ 

We will use this generating rule to obtain a formula for the generating function 

M(t,u,v) :=J2 Yl u eM v rM t n . 

n>l 7reS„(2-l-3,12-3) 

For fixed £ and r, let M i>r (t) = £„>i \{n £ <S„(2-l-3, 12-3) : = £,r{n) = r}\ t n . Note that 

M(t,u,v) = J2 er Mt r (t)u e v r . 

Proposition 3.2. The generating function for {2-1-3, 12-3} -avoiding permutations where u and v mark 
the parameters £ and r defined above is 

_ [(1 - u)v + cit + c 2 t 2 + c 3 t 3 + c 4 t 4 - ((1 - u)v + tu + t 2 u 2 v)Vl - 2t-U 2 )]u 2 v 
( t,U,V > ~ 2(1 - u - tu(l -u)+ t 2 u 2 )(l -uv + two + t 2 u 2 v 2 ) ' 

where c\ = 2 — u — v — uv + 2u 2 v, c 2 = u(— 1 + (2 — u)v + 2(u — l)v 2 ), C3 = u 2 v(— 3 + 2v — 2uv), and 
c 4 = —2u 3 v 2 . 

Substituting u = v = 1 in the above expression we recover the generating function for the Motzkin 
numbers. 

Proof. The coefficient of t n in M(t, u, v) is the sum of u l v r over all the pairs (£, r) of labels that appear 
at level n of the tree. By Lemma 13.11 the children of a node with labels (£, £) contribute u l+1 v + 
u e+1 v 2 + • ■ • + u t+1 v l to the next level, and the children of a node with labels (£, r) with £ > r contribute 
u e+1 v + u e+1 v 2 + ■■■ + u e+1 v r + u r+1 v r+1 . It follows that 

(6) M{t,u,v) =tu 2 v+t^Mw{t)u t+1 {vW + - ■ ■W)+t^M t , r {t)[u t+ \vW + - ■ ■ + 

I l>r 

It will be convenient to define 

M>(t,u,«):=^ J2 u^v rM t n and M=(t, u, v) := ^ ^ {uv) l{ ^ t n , 

n>l 7reS„(2-l-3,12-3) n>l 7reS„(2- 1-3, 12-3) 

with £(7r)>r(7r) with li^n)-r(-n) 

so that M(t, u, v) — My (t, u, v) + M = (t, u, v). Taking from © only the pairs (£, r) with £ > r, we get 
M y (t, u, v) = tu 2 v + t ^2 M^(t)u e+1 {v + v 2 + ■ ■ ■ + v e ) + t^2 M^ r (t)[u e+1 (v + v 2 + ■■■ + v r )} 

l l>r 

= tu 2 v + t Y M e t (t)u l+l vW "~ - + 1 Y M e Jt)u i+l Vr+1 ~ V 
^ — J ' v — 1 / — 1 ' v — 1 

l £>r 

(7) = tu 2 v + -!^-\M = (t,u,v) - M=(t,u, l) + M > (t,u,v) - M > (t,u, 1)1. 

v — 1 

Similarly, taking from ^ only the pairs {£, r) with £ = r, 

(8) M = (t,u,v) =ty^M eir (t)u r+1 v r+1 =tu v y j M^ r {t){uv) r = tuv M>(t,l,uu). 

l>r t>T 

Using in (|7|) the expression of M— in terms of M> given in (8]), we get 

(9) M>(*,u,v) = tu 2 v H [tuv M>(t, l,uv) - tu M>(t, l,u) + M>(t, it, u) — M> (t, u, 1)] . 

v — 1 

Substituting it = 1 in this equation and collecting the terms in M > {t 1 1, w), we have 

do) f i - - M >^ x < u ^ ^ - ~^~~t~ m > (*. 1. 1)- 

\ u — 1 u — 1 / v — 1 



s 
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Now we apply the kernel method, substituting v = v = v (t) = 1 1 v 2 1 f2 2t 3t2 m (HOI) to obtain 



M>(t,l,l) = 



2t 2 

v - 1 1 - t - 2t 2 - Vl-2t- 3t 2 



t+1 2t 2 {t+l) 
Plugging this expression for M>(i, 1, 1) back into (fTO]) we get that 

, „ , (1 - t - 2t 2 v - yjl - 2t - 3t 2 )v 
2t{\ — v + tv + t z v £ ) 

If we write equation © as 

/ tuv \ tuv 

(12) 1 M > (t,u,v) = tu 2 v H [tuv M > (t, l,uv) - tu M > (t, l,u) - M>(t,u, 1)1 , 

\ v — 1 / u — 1 

we can apply again the kernel method, taking v — v\ = Vi(t, u) = l \ u ■ This cancels the left hand side 
and gives 

[2(1 ~u)+u 2 - t(l + 2t)u 2 + (1 - 2u)Vl -It- 3t 2 )]tu 2 
>(< '"' >~ 2(1 - u + tu + t 2 u 2 )(l -u- tu{\ -u)+ t 2 u 2 ) 

using (jlip . Substituting back into (fT2"|) and using pT|) again we get that 

[2-u-uv + u 2 v + tu(v - 1) - i(l + 2t)u 2 v + (1 - 2u)Vl - It- U 2 )\tu 2 v 



M > (t,u,v) = 



2(1 -u- tu(l - u) + t 2 u 2 )(l - uv + tuv + t 2 u 2 v 2 ) 
Finally, combining it with the fact that 

M(t,u,v) = M > (t,u,v) + M = (t,u,v) = M > (t,u,v) + tuv M > (t, l,uv), 
we obtain the desired expression for M(t, u,v). □ 

We have encountered two classes of pattern-avoiding permutations enumerated by the Motzkin num- 
bers, namely 

(13) \S n+1 (2-1-3, 2-31)| = |5„(2-l-3, 12-3) | = M n 

(see Proposition ^. ip . In Section [2. II we described a bijection <p between 2-3-1-avoiding permutations and 
Dyck paths. A permutation ir is 2-31-avoiding if and only if (p(ir) is a UDU-hee Dyck path. It is not 
hard to check (see [TT]) that ir is 12-3-avoiding if and only if <p(tt) is UUU-hee. Next we give a bijection 
between UDU-free Dyck paths of size n+1 and UUU-hee Dyck paths of size n, reproving equation (fT3^) . 

Given a UDU-iree Dyck path, mark each D that is immediately preceded and followed by a D (and 
also the rightmost D if it is preceded by a D) . Move left each one of the marked Ds so that it immediately 
follows its matching U. Finally, delete the rightmost peak (i.e., occurrence of UD). This gives a DDD- 
free Dyck path, which can be easily turned into a UUU-iree one by reversing it, that is, reading the steps 
from right to left and exchanging ?7s and Ds. 

To show that this map is the desired bijection, we now describe its inverse. Given a DDD-iree Dyck 
path, we first reverse it and then append a peak UD to it. Define the height of a step to be the number of 
Us minus the number of Ds preceding it. Mark each D step in an occurrence of UDU. For each marked 
step, if h is its height, move it to the right so that it immediately precedes the next D step with height 
h — 1 (if ft = 1, then move it to the end). This produces the original UDU-iree Dyck path. 

As an example of this bijection, consider the UDU-free path UUUUDDUUDDDDUUDD. The 
marked Ds and their matching Us are distinguished with a bar. The DDD-iree Dyck path that we 
obtain is UU DUU DDU DU DDU D (the barred Ds are the steps that have been moved), and its reversal 
is U DUU DU DUU DDU DD . Applying the inverse map moves the barred Ds back to their original 
position. 
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3.2. {2-1-3, 32-l}-avoiding permutations. It is known [8] that |<S n (2-1-3, 32-1) | = 2™ . Here we use 
rightward generating trees with two labels to recover this fact, and to refine it with two parameters: the 
value of the last entry and the largest value of the bottom of a descent. Given any tt G S n , define 



h(ir) 



if 7T = 12 • 

max{7Ti : i > 1, 7T;_i > Hi] otherwise. 



To each {2-1-3, 32-l}-avoiding permutation ir we assign the pair of labels (h,r) = (h(ir), r(n)). Note 
that since 7r avoids 32-1, then necessarily h < r. 

Lemma 3.3. The rightward generating tree for {2-1-3, 32-1} -avoiding permutations is specified by the 
following succession rule on the labels: 

(0,1) 

{h, r) — ► (h + 1, h + 1) (h + 2, h + 2) • • • (r, r) (h, r + 1). 

Proof. When we append an entry i to a {2-1-3, 32-1 {-avoiding permutation, the new permutation is 2-1-3- 
avoiding if and only if i < r(n) + 1, and it is 32-1-avoiding if and only if i > h(ir). The list of labels of the 
children obtained by appending an i satisfying these two conditions is the right hand side of the rule. □ 



)u h v r . 



Let 

n>l 7reS„ (2-1-3,32-1) h,r 

Proposition 3.4. The generating function for {2-1-3, 32-1}- avoiding permutations where u and v mark 
the parameters h and r defined above is 

tv(\ — t + tu — tuv) 



N(t,u,v) 



(1 - tv)(l - t - tuv)' 



Proof. By Lemma T3.3[ the children of a node with labels (h, r) contribute u h+1 v h+1 + u h+2 v h+2 + ■ • ■ + 
u r v r + u h v r+1 to the next level. It follows that 



N(t,u,v) = tv + t^2N h:r (t) 



h.r 



+ u h v r+1 



(uv) r+1 - (uv) h+1 
uv — 1 

tuv[N(t, l,uv)- N(t,uv, 1)] 



(14) = tv + tvN(t,u,v) , 

uv — 1 

Substituting u = 1 and v = 1 separately gives a system of two equations in N(t, 1, *) and N(t, *, 1) 
that can be easily solved. □ 

The above result can indeed be obtained as well without using rightward generating trees. The recursive 
structure of 2-1-3-avoiding permutations (i.e., they are of the form ctIt, where a and r are 2- 1-3- avoiding 
and every entry in a is larger than every entry in r) can be used to obtain an equation satisfied by 
N(t, u, v) and to deduce the above formula without much difficulty. 

3.3. {2-1-3, 34-21 {-avoiding permutations. The labels that will be convenient to use to describe the 
rightward generating tree for this class are (s,r) = (s(ir), r(jr)), where 



(15) s(tt) 



if 7T = n(n- 1) • • -21, 

max{7T; : 7r,; < TTi+i} otherwise, 

and r(7r) = ir n as usual. 

Lemma 3.5. The rightward generating tree for {2-1-3, 3A-21} -avoiding permutations is specified by the 
following succession rule on the labels: 

(0,1) 

' (s + 1, 1) (s + 1,2) ■ • • (s + 1, s) (s, s + 1) (r, r + 1) if s < r, 
(s + 1, r + 1) if s > r. 



(s,r) 
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Proof. First note that the 2-1-3-avoiding condition implies that the new entry appended to ir has to be 
at most r + 1. If s < r and iti is an entry to the right of s, then 7T; > s, otherwise s^r would be an 
occurrence of 2-1-3. In fact, we also know that tti ^ s + 1, unless iti = r = s + 1, because otherwise the 
entry following s + 1 would be greater than it, contradicting the definition of s. So, unless r = s + 1, the 
entry s + 1 precedes s, so the appended entry cannot be greater than s + 1, otherwise it would create a 
2-1-3. This explains the labels in the case s < r. If s > r, the appended entry has to be greater than r 
for the new permutation to be 34-21-avoiding. □ 

Let 

n>l 7TG5„(2- 1-3, 34-21) s,r 

and let K < (t,u,v) and K > (t,u,v) be defined similarly, with the sum running only over permutations 
with s(ir) < r(n) and s(ir) > r(7r), respectively, so that K{t,u,v) = K < (t 1 u,v) + K > {t, u, v). 

Proposition 3.6. The generating junction for {2-1-3, 34-21} -avoiding permutations where u and v mark 
the parameters s and r defined above is 

_ , tvll - (1 + u + uv)t + (u 2 + uv + u 2 v)t 2 ] 
(16) K (i, u, v) = 



(1 - t - tu)(l - 1 - tuv)(l - tuv) 
Proof. By Lemma 13.51 the generating functions K < and K > satisfy 

K<{t, u, v) = tv + t^2K s , r (t)(u s v s+1 + u r v r+1 ) =tv + tv[K<(t, uv, 1) + K<{t, 1, uv)], 
K^u.v) = t^K s , r {t)u s+1 {v + --- + v s )+t^K s , r {t)u s+1 v r+1 



s < r s > r 

tuv 



V-l 



[K^uv, 1) - K < (t,u, 1)] +tuvK > (t,u,v). 



Substituting first u = 1 and then v = 1 in the first equation, we get two equations involving K < (t, 1, w) 
and K K (t, w,l) that can be easily solved to give 

. . tv 
K<{t,u, v) 



1 — t — tuv 
The second equation then implies that 

u 2 vt^ 

K>{t,u,v) - ^ - - - - - - - tuv ^ , 

and the proposition follows. □ 

Corollary 3.7. The number of {2-1-3, 34-21} -avoiding permutations of size n is (n — 1)2"~ 2 + 1. 
Proof. Taking u = v = 1 in (fTB|) . we get that 

y ' ' ' (l-f)(l-2t) 2 

The coefficient of t n in the series expansion of this rational function is (n — 1)2™~ 2 + 1. □ 



It is not hard to show that S n (2-1-3, 34-21) = S n (2-1-3, 3-4-2-1) = S„( 1-3-2, 3-4-2-1). This last set of 
permutations was enumerated by West [TS], and Corollary 13 . 71 agrees with his result. 
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3.4. {1-2-34, 2-l-3}-avoiding permutations. The generating function for these permutations appears 
in [15] . In fact, it is easy to see that S n (1-2-34, 2-1-3) = S n (1-2-3-4, 2-1-3), and the latter set of permuta- 
tions was studied in [18], where it is shown that they are counted by the Fibonacci numbers fbn-i- Here 
we derive the generating function and obtain a refinement of it using a rightward generating tree with 
two labels. 

Let the labels of a permutation 7r be the pair (m, r) — (m(ir), r(ir)), where r(ir) — n n and 



n+l if 7r = n(n- 1) • • -21, 

min{7Ti : 3j < i with ttj < iri} otherwise. 



Note that we always have m(7r) < r(n) unless r = 1, and that if m(jr) — r(7r), then tt = n(n — 1) • • • 312, 
so to = r = 2. 

Lemma 3.8. The rightward generating tree for {1-2-34, 2-1-3} -avoiding permutations is specified by the 
following succession rule on the labels: 

(2,1) 

f(m + 1,1) (2,2) tfr=l, 
(m,r)— J (3,1) (2,2) (2,3) ifm = r = 2, 

y (m +1,1) (2, 2) (to, to + 1) ■ • • (to, r) if m < r. 

Proof. As usual, the appended entry has to be at most r + 1 for the permutation to avoid 2-1-3. In 
the case that to < r, this entry cannot be greater than r in order to avoid 1-2-34. The labels are now 
obtained by looking at how the parameter to changes after appending the new entry. □ 

Let H(t, u, v) := J2n>i E7reS„(l-2-34,2-i-3) u m(7r V (7r) t n , and let Hi(t, u, v), H = (t, u, v), and H < (t, u, v) 
be defined similarly, with the summation restricted to permutations with r(jr) — 1, to(tt) = r(w), and 
to(7t) < r(7r), respectively, so that H (t, u, v) = H\(t, u, v) + H = (t, u, v) + H < (t, u, v). 

Proposition 3.9. The generating function for {1-2-34, 2-1-3} -avoiding permutations where u and v mark 
the parameters to and r defined above is 

TT , . tu 2 v[l + (v - 3)t + (1 + u - v - uv + v 2 )t 2 + uv(l - v)t 3 } 

Hit.U.V) = ; 77-—: ; . 

V ' ' ; (1 - 3t + t 2 )(l - tu) 

Proof. From Lemma 13.81 we get the following functional equations defining Hi, H-, and H < . 

(17) Hi(t,u,v) = tu 2 v + tuvH(t,u,l) 

(18) H=(t,u,v) = tu 2 v 2 H(t, 1,1) 

tv 

(19) HAt,u,v) = tvH = (t,u,v) + [HAtiUiv) - HAt^uv,!)} 

v — 1 

Combining (|18[) and p9p . introducing a variable w — uv, and defining H < (t,w,v) = -ff<(i, —,v), we get 

/ tv \ ~ tv 

(20) 1 H< (t, w, v) = t 2 vw 2 H(t, 1, 1) -H<(t, w, 1). 

\ v — 1 / v — 1 

The kernel is canceled with v = jzrt, giving an expression for H < (t, w, 1) in terms of H(t, 1, 1), which 
plugged back into (|20]) yields 

t 2 u 2 v 3 

(21) HAt,u,v) =H < (t,uv,v) = -j—^H(jt,l,l)- 

On the other hand, adding equations (fTT)) and (JT5J) and using that Hi (t, u, v)+H— (t, u, v) — H(t, u, v) — 
H < (t, u, v), we get 

H< (t, u, v) = H{t, u, v) - tu 2 v - tuvH(t, u, 1) - tu 2 v 2 H{t, 1,1), 
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which combined with (j2Tj) gives a simple expression relating H(t, u, v), H(t, u, 1) and H(t, 1, 1). In this 
expression, the substitution u = v = 1 gives 

*(m,u= 1(1 -" 



1 - 3t + t 2 ' 

and the substitution v = 1 puts if (i, it, 1) in terms of _ff (t, 1, 1). All together produces the desired formula 
for H(t,u,v). □ 

3.5. {12-34, 2-l-3}-avoiding permutations. It was proved in [TS] that the generating function for per- 
mutations avoiding {12-34,2-1-3} is i-2t-t 2 -Vi-4t+2t^+t^ Uging the labelg defined as in ^ we 
can construct a generating tree with two labels for this class of permutations. The proof of the following 
lemma is straightforward and analogous to that of Lemma 13.11 

Lemma 3.10. The rightward generating tree for {12-34, 2-1-3} -avoiding permutations is specified by the 
following succession rule on the labels: 

(2,1) 

((£ +1,1) (f+1,2) ••• (l + l,r) ( r + l,r + l) if £ > r, 

(£,r)~ > I (£+1,1) (1+1,2) ■■■ (1 + 1,1) (£,£+1) ifl = r, 

((£+1,1) (£+1,2) ••• (£ + !,£) (£,£+1) (£,£ + 2) ••• (£,r) if£<r. 

Let F(t,u,v) := Y. n >i X^ e s n (12-34,2-1-3) u*Mu r M t n , and let F > (t,u,v), F = (t,u,v), and F<(t,u,v) 
be defined similarly, with the summation restricted to permutations with £(ir) > r(7r), £(ir) = r(n), and 
£(n) <r(n), respectively. By definition, F(t,u,v) = F > (t, u, v) + F=(t, u, v) + F<(t, u, v). 

Proposition 3.11. The generating function for {12-34, 2-1-3} -avoiding permutations where u and v 
mark the parameters I and r defined above is 



_ u 2 v[ Pl (t, u, v) + p 2 (t, u, v)Vi -4t + 2t 2 + t 4 ] 
[,u,v)~ 2 [(i + tuv y -uv-t-uvt 2 ][l + (u + t)(tu- 1) 



where 



pi(t, u, v) = (1 — u)v + (2 — u — 4v + 2uv + v 2 + 2u 2 v — uv 2 )t + (—4 + u + 6v + uv — 3v 2 — 6u 2 v + 3u 2 v 2 )t 2 
+ (2 + u - 4v - 5uv + 3v 2 + 4u 2 v + 4uv 2 - 4u 2 v 2 - 2uv 3 - 2uv 2 + 2u 2 v 3 )t 3 
+ (—u + v + 4uv — v 2 — 4uv 2 — u 2 v 2 + 2uv s + 2u i v 2 — 2u 3 v 3 )t 4 — uv(v — l)(2uv — l)t 5 , 

Pi(t, u, v) = (u — VjV + [(u — l)v(v — 2) — u]t + (u — v + v 2 — u 2 v 2 )t 2 + uv(l — v)t 3 . 

Note that this expression becomes much simpler if we ignore the parameter r, that is, 



_ u 2 (l - 2tu - t 2 - Vl - 4i + 2t 2 + t A ) 
{t ' U ' 1} ~ 2{l + (u + t)(tu-l)} ' 

and coincides with the result from [15j if we ignore both parameters: 



1 - 2t - t 2 - VI - At + 2t 2 + t* 
F(t, 1, 1) — 2 . 

Proof. From Lemma 13.101 we get the following functional equations defining F> , F— , and F < . 

(22) F>(t,u,v) = tu 2 v+-^-\F > (t,u,v)-F > (t,u,l) 

v — 1 

+ F = (t, u, v) - F = (t, u, 1) + F< (t, uv, 1) - F< (t, u, 1)], 

(23) F = (t,u,v) = tuvF y (t,l,uv), 

tv 

(24) F<(t,u,v) = tvF = (t,u,v) + —j[F < (t,u,v)-F < (t,uv,l)}. 
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We can introduce a variable w — uv in (|24j) , and apply the kernel method with v = tzt to get that 

F<(t,w,l) = — . 

Using this expression together with ([23]) in (f22|) , we get an equation that involves only F> : 



tuv \ „ , . „ . 2 tuv 



„ / x tuv „ , , tu „ . 

F>(t, u, i) + y— ^> (*> x > H - 737 > ( *' ' u) 



(25) 1 -)F > {t,u,v) = tu'v- 

\ v — 1 J v — 1 

Substituting u = 1, it becomes 

, , (1 - v - 1 + 2tv - vt 2 + v 2 t 2 ) , . to , „ , 

26 ^ — — L F> (t,l,v) = tv+- -—-i^t, 1,1). 

(l-v)(l-t) (l-v){l-t) 

We apply the kernel method again, this time with v = 1 ~ 2t+t -V^i-4t+2t 'f_+t± ^ Q cance j ^he \ e ft nan( j gidg 
of ([26]) . which yields 



„ , ... 1 - 3t + + i 3 + (t - 1) VI - 4t + 2t 2 + 1 4 
F>(*,1,1) = ^ ■ 

Now we can use (|26p to obtain a formula for F>(i, 1, w). Applying again the kernel method in ([23]) . with 
i> = tzt^j we S e t an expression for F>(t, u, 1) in terms of F > (t,l,u) and F > (t, 1, t " ), and therefore 
a formula for F > (t,u, 1). Substituting back into (|2"5"|) , we get a formula for F > (t, u, v). From this it is 
straightforward to obtain formulas for F < {t, u, v) and F—(t, u, v) as well, and the result follows. □ 

3.6. {1-23, 3-12}-avoiding permutations. Generating trees with two labels can be used to obtain the 
generating function for the number of {1-23, 3-12}-avoiding permutations. These permutations were 
studied in [S], where it was shown that if we let b n = |iS n (l-23, 3-12)|, then these numbers satisfy the 
recurrence b n+ 2 = b n +i + Y^k=o (fc)^fc- Here we obtain an ordinary generating function without going 
through the recurrence. The labels are particularly easy in this case because we can take one of them to 
be just the length n of the permutation. The labels of tt G S n are then (r, n), where r — ir n as usual. 
The advantage of having one of the labels be n is that we do not need an extra variable for this label in 
the generating function, since it is already encoded in the exponent of the variable t. 

Lemma 3.12. The rightward generating tree for {1-23, 3-12} -avoiding permutations is specified by the 
following succession rule on the labels: 

(1,1) 

f(l,n + l) (n + l,n + l) ifr = l, 

[ (l,n+l) (2,71+1) ••• (r,n + l) ifr>\. 



(r, n) 



Proof. The appended element cannot be larger than the rightmost entry of tt, except where this entry is 
1, in which case the appended element can be the new largest one. □ 

Let P(t, U ) := £„>1 E^d-23,3-12) ^ W ^ = ZrMtK- 

Proposition 3.13. The generating function for {1-23, 2>-\2\- avoiding permutations is 

P{t l) = y t"- 1 (i-(*-Pt) 

Proof. From Lemma 13.121 we get 

tv 

(27) P(t,u) =tu+ ~{P{t,u) - uPj.it) - Pit, 1) + PAt)) + tu(Pi(t) + Pi{tu)). 

u — 1 

Using that Pi(t) = t + t P(t, 1) and collecting the terms with P(t, u), we get 

(28) (l - -^j) P(t, u)=tu + t 2 u 2 + t 2 u 2 P(tu, 1) + (t 2 u + M<__L_MA p(tj 1} . 
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Substituting u — — gives 



p(t,l) = _±_f 1 + iP( * 1)), 



(i-t) 2 V i-* 

and by iterated application of this formula, 

[h ) (1 - t)2 \ + (1-24)2 ^+ (i _ £ )(i _ 3t) 2 V (l-2i)(l-4i)2 11 + j 
f rj 3 /j 5 i 7 



(1-t) 2 (1 - t)(l - 2t) 2 (1 - t) 2 (l - 2t)(l - 3i) 2 (1 - t) 2 (l - 2t) 2 (l - 3i)(l - 4t) 2 



which is the formula above. If we substitute this expression back into [[28]) we get the refined formula for 
P(t,u). □ 

A very similar argument can be applied to 1-23-avoiding permutations, which are known to be enu- 
merated by the Bell numbers [8] . Our approach in this case gives essentially the same functional equation 
that is derived in [13] using what the authors call the scanning- elements algorithm. 

Rightward generating trees and the kernel method can also be used to produce a functional equation 
for the ordinary generating function of 123-avoiding permutations. We omit this result here because a 
more direct way to enumerate these permutations was already given in [12] , 

4. Generating trees with three labels 

In this section we include two instances of permutations avoiding generalized patterns where the 
rightward generating tree can be described by a succession rule with three labels. One of these labels 
is the length of the permutation, so that the functional equations that we obtain have three variables 
instead of four. However, the fact that the variable t appears multiplied by another variable adds some 
difficulty to the equations. 

To the best of our knowledge, the two classes of restricted permutations considered in this section have 
never been enumerated before. 

4.1. {1-23, 3-12, 34-21}-avoiding permutations. Given a permutation tt 6 S n , let s(ir) be defined as 
in (fT5|) . We associate to ir the triple of labels (s, r, n) — (s(tt), r(7r), n). 

Lemma 4.1. The rightward generating tree for {1-23, 3-12, 3 A-21} -avoiding permutations is specified by 
the following succession rule on the labels: 

(0,1,1) 

'(s + l,l,n+ 1) (s + l,2,n+l) ••• (s+ 1,5,71+1) 

(s, s + 1, n + 1) (s, s + 2, ri + 1) • • • (s, r, n + 1) if s < r ^ 1, 

(s, r,n) — > { (0, 1, n + 1) (1, n + 1, n + 1) if (s, r) = (0, 1), 

(s,n + 1,71+1) if s > r = 1, 

if s > r > 1. 

Proof. If r > 1, the appended entry has to be at most r for the new permutation to avoid 1-23. If s > r, 
it has to be at least r + 1 for the new permutation to avoid 34-21. Finally, if r = 1, the appended entry 
has to be n + 1 for the permutation to avoid 3-12, unless s — 0, which means that n is the decreasing 
permutation. Combining these conditions we get the four possible cases and the new labels in each 
case. □ 

The four cases in the succession rule above suggest dividing the set O of values that the pair (s, r) 
can take into four disjoint sets: 6i = {(s,r) : s < r ^ 1}, 2 = {(0,1)}, 63 = {(s,r) : s > r = 1}, 
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6 4 = {(s,r) : s > r > 1}. For i = 1,2,3,4, let 

Ri{t,u,v) :=J2 E u sM v rM t n , 

n>\ 7r£5„(l-23, 3-12, 34-21) 
with (s(7r),r(7r))e0 i 

and let i?(*, u, v) = i?i(i, u, v) + R 2 {t, u, v) + i?3(*, u, v) + R^t, u, v). 

Proposition 4.2. The generating function for {1-23, 3-12, 34-21} -avoiding permutations where u marks 
the parameter s defined above is 



t 2k u k (l + ktu) 

Proof. Lemma |4 . 1 1 translates into the following equations for the generating functions Ri\ 

tv 

Ri(t,u,v) — tuvR 2 (tv, 1, 1) + tvR 3 (tv, u, 1) H -\Ri(t, u, v) — R\(t, uv, 1)1, 

v — 1 

tv 

R 2 (t,u,v) = 

(30) Rs(t,u,v) = tuvRi(t,u,l), 

tuv 

(31) R 4 (t,u,v) = \R 1 (t,uv,l)-vRx(t,u,l)]. 

v — 1 



Combining them we get an equation involving only R 



Ri{t, u, v) = Vtuv Ri(tv, u, 1) H [Ri{t, u, v) — i?i (t, uv, 1)1. 

1 — tv v — 1 

If we collect on one side the terms with Ri(t, u, v), the kernel of the equation is 1 — -3=-. Introducing a 
new variable w — uv and canceling the kernel with v — jzj, we obtain an expression involving Ri(t,w, 1) 
and i?i(y3j, (1 — t)w, 1), which can be simplified to 



Ri(t,w,l) = tw 2 



1 ft,, 
R l -,(l-t)w,l 



1 — 2t 1 - 1 V 1 - * 



By iterated application of this formula, 

, , 1 * 2 u ( 1 * 2 u / 1 t 2 u ( 1 

Ri(t,u, 1) = *^it 



1 — 2* 1 — * V 1 — 3 * 1-2* V 1 - 4 * 1 - 3t V 1 - 5i 
* 2 u (* 2 u) 2 {t 2 uf (t 2 u) 4 



1-2* (l-*)(l-3*) (l-*)(l-2*)(l-4*) (1-*) (1-2*) (1-3*) (1-5*) 
t 2fe w fc 



E 



fe >! a-(fc +i)t)nK(i-i*)' 

Equation (|3"0|) gives now an expression for R^(t,u, 1), and (|3"Tj) implies that 

^2fc+l u fc+l( w 2 +w 3 + ... +w fe ) 



R 4 (t,u,v) = 22 



fc >i (i-(fc+i)*)nK(i-iO ' 

Adding up the four generating functions i?(*, u, 1) = Ri(t, u, 1) + i?2(*, u, 1) + i?3(*, u, 1) + Ri{t, u, 1) we 
get (22). □ 

The first coefficients of i?(*, 1, 1), which are the values of |<S„({l-23, 3-12, 34-21}) | for n = 1,2, . . ., are 
1, 2, 4, 8, 19, 47, 125, . . .. This sequence does not appear in [T^] at the moment. 
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4.2. {1-23, 34-21}-avoiding permutations. The derivation of the generating function for this class of 
permutations is very similar to the previous subsection. The labels that we associate to a permutation 
are again (s, r, n). The proof of the next lemma is analogous to that of Lemma T4. II 



(s,r,n) 



Lemma 4.3. The rightward generating tree for {1-23, ZA-21}- avoiding permutations is specified by the 
following succession rule on the labels: 

(0,1,1) 

*0+ (s+ l,2,n+l) ••• (s + l,s,ra+l) 

(s, s + 1, n + 1) (s, s + 2, ri + 1) • • • (s, r, n + 1) ifs<r^l, 

(0,l,7i + l) (l,2,7i + l) (l,3,7i + l) ••• (l,n+l,7i+l) z/(s,r) = (0,1), 
(s+ 1,2,71+1) (s + 1,3, 71+1) ••• (s + l,s,?i+l) 

(s, s + 1, ?i + 1) (s, s + 2, 7i + 1) • • • (s, 7i + 1, 7i + 1) ifs>r = l, 

if s > r > 1. 

Divide the set of values that the pair (s, r) can take into four disjoint sets ©i, z = 1,2,3,4 as before, 
and let 

Ti{t,u,v):=Y^ Yl u s{7r) v rM t n 

n>l TreS n (1-23,34-21) 
with (s(7r),r(7r))e6; 

and T(t, u, v) — T\{t, u, v) + T 2 (t, it, v) + T 3 (i, u, v) + T 4 (i, u, v). 

Proposition 4.4. The generating function for {1-23, 3A-21} -avoiding permutations where u marks the 
parameter s defined above is 

t k+1 u k (l + ktu) 



(32) 



T 



k>Q 



(1 + tu) k (l - kt)(l - (k + l)t) 



Proof. The equations that follow from Lemma l4.3l are now 

Tx(t,u,v) 



-[T 2 (tv, 1, 1) - T 2 (t, 1, 1)] + -[vT 3 (tv, u, 1) - T 3 {t, uv, 1)] 

v — 1 v — 1 



(33) 
(34) 



T 2 (t, u, v) 
T 3 (t,u,v) 
T 4 (t,u,v) 



+-^-[T 1 (t,u,v)-T 1 (t,uv,l)}, 
v — 1 

tv 

I - ? 

tuvT\{t 1 u, 1), 
tuv 



v-1 



[Ti(/j,U7j, 1) +T 3 (/j,it7j,l) - vTi(t, u, 1) - uT 3 (t, 7j, 1)]. 



From them we can get an equation involving only T\ 



Tx(t,u,v) 



t 2 uv 2 



1 



t uV tv 

+ -[«T 1 (tt;,u,l)-T 1 (t,ut;,l)] + -[Ti («,«,«) - r x (t, uv, 1)]. 

7J — 1 V 1 — tV 1 — t J V — 1 7J — 1 



Letting w = uv and canceling the kernel with tj = y^r , we get that 



Ti(t,™,l) 



Iterating this formula, we see that 



1 - It 



1 - f 



,(1 -«)«;,! 



T 



i(m,i) = E?i 



t k+i u k 



k>l 



(l + tu) k (l-kt)(l- (k + l)t)' 



Using (f3"3")) and (|3~4")) we get expressions for T 3 (t,u, 1) and T±{t,u,v). Finally, the sum T(t, u, 1) = 
Ti(i,ii, l)+T 2 (t,K, 1)+T 3 (*,u, 1)+T 4 (i,u, 1) gives the formula ([21]). □ 
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The first coefficients of T(t, 1, 1) are 1, 2, 5, 14, 42, 138, 492, . . ., which teaches us not to judge a se- 
quence by looking only at its first five terms. This sequence gives the number of {1-23, 34-21 }-avoiding 
permutations of size n — 1, 2, . . ., and does not currently appear in [16] . 

5. Concluding remarks 

The main results in the paper have been obtained by constructing rightward generating trees with up 
to three labels for several families of pattern- avoiding permutations, and solving the functional equations 
for the generating functions that the succession rule produces. This is a useful method for enumerating 
permutations avoiding generalized patterns. There is nothing special about the sets of patterns studied 
in this paper, except that this method happens to work out nicely on them. 

We expect that this technique of rightward generating trees with several labels, together with the 
kernel method and other ad-hoc tools for solving the functional equations that are obtained, will lead to 
many more enumerative results for classes of permutations avoiding generalized patterns. 

Acknowledgements. I am grateful to Mircillc Bousquet-Melou for many helpful ideas that have made 
this paper possible, and to two anonymous referees for useful suggestions to improve its presentation. 
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